Lecture 05
Lecture 4: Review
- Introduction to hypothesis testing
- The standard normal distribution
- Standard error
- Confidence intervals
- Student’s t-distribution
- H testing sequence
- p-values
Our last graphs
Lecture 5: Overview
The objectives:
- p-values
- Brief review
- H test for a single population
- 1- and 2-sided tests
- Hypothesis tests for two populations
- Assumptions of parametric tests
asdfasfasd
Lecture 5: Statistical hypothesis testing
- Major goal of statistics:
- inferences about populations from samples…
- assign degree of confidence to inferences
- Statistical hypothesis testing:
- formalized approach to inference
- Hypotheses ask whether samples come from populations with certain properties
- Often interested in quewstions about population means
- but other questions are of interst
- inferences about populations from samples…
asdfasfasd
Lecture 5: Statistical hypothesis testing
Useful hypotheses: - Rely on specifying - null hypothesis (Ho) - alternate hypothesis (Ha)
- Ho is the hypothesis of “no effect”
- two samples from population with same mean
- sample is from population of mean = 0
- Ha (research hypothesis)
- is the opposite of the Ho
- or predicts that there is an effect of x on y
- but does NOT suggest a direction
asdfasfasd
Lecture 5: Statistical hypothesis testing
Together Ho and Ha encompass all possible outcomes: - For Example:
- Ho: µ=0, Ha: µ ≠ 0 - mean equals 0 or mean does not equal 0 - Ho: µ=3700, Ha: µ ≠ 3700 - mean equals 3700 or mean does not equal 3700 - Ho: µ1 = µ2, Ha: µ1 ≠ µ2 - mean of population 1 equals mean of population 2 or it does not - Ho: µ > 0, Ha: µ ≤ 0 - can be directional mean is greater than 0 or mean is not equal or less than 0
asdfasfasd
Lecture 5: Statistical hypothesis testing
Tests assess likelihood of the null hypothesis being true
- If the Ho is likely false, then Ha assumed to be correct
- More precisely:
- the long run probability of obtaining sample value (or more extreme one) if the null hypothesis is true
- p(data|Ho) - the probability of observing the data given that the null hypothesis Ho is true
- the long run probability of obtaining sample value (or more extreme one) if the null hypothesis is true
asdfasfasd
Lecture 5: Statistical hypothesis testing
Hypothesis tests
- Expressed as p-value (0 to 1)
- Interpret p-value as:
- probability of obtaining sample value of statistic (or more extreme one) if Ho is true
- High p-value:
- high probability of obtaining sample statistic under Ho
- if the null hypothesis (Ho) were true, you would frequently observe data similar to or more extreme than your sample statistic
- your observed results are quite compatible with what the null hypothesis predicts
- low p-value: low probability of obtaining sample statistic under Ho
- if the null hypothesis (Ho) were true, you would rarely observe data similar to or more extreme than your sample statistic
- Your results are unusual under the null hypothesis, suggesting that either you’ve witnessed a rare event or the null hypothesis may be incorrect
- high probability of obtaining sample statistic under Ho
asdfasfasd
Lecture 5: Statistical hypothesis testing
Statistical test results: - p = 0.3 means that if I repeated the study 100 times, I would get this (or more extreme) result due to chance 30 times - p = 0.03 means that if I repeated the study 100 times, I would get this (or more extreme) result due to chance 3 times
Which p-value suggests Ho likely false?
asdfasfasd
Lecture 5: Statistical hypothesis testing
Statistical test results:
At what point reject Ho? - p < 0.05 conventional “significance threshold” (α = alpha or p value) - p < 0.05 means: - if Ho is true and we repeated the study 100 times - we would get this (or more extreme) result less than 5 times due to chance
asdfasfasd
Lecture 5: Statistical hypothesis testing
Statistical test results: - α is the rate at which we will reject a true null hypothesis (Type I error rate) - Lowering α will lower likelihood of incorrectly rejecting a true null hypothesis (e.g., 0.01, 0.001)
*Both Hs and α are specified **BEFORE collection of data and analysis*
asdfasfasd
Lecture 5: Statistical hypothesis testing
Traditionally α=0.05 is used as a cut off for rejecting null hypothesis
There is nothing magical about 0.05 - actual p-values need to be reported - also need to decide prior to study
| p-value range | Interpretation |
|---|---|
| P > 0.10 | No evidence against Ho - data appear consistent with Ho |
| 0.05 < P < 0.10 | Weak evidence against the Ho in favor of Ha |
| 0.01 < P < 0.05 | Moderate evidence against Ho in favor of Ha |
| 0.001 < P < 0.01 | Strong evidence against Ho in favor of Ha |
| P < 0.001 | Very strong evidence against Ho in favor of Ha |
Lecture 5: Statistical hypothesis testing
Lecture 5: Statistical hypothesis testing
Fisher:
p-value as informal measure of discrepancy betwen data and Ho
“If p is between 0.1 and 0.9 there is certainly no reason to suspect the hypothesis tested. If it is below 0.02 it is strongly indicated that the hypothesis fails to account for the whole of the facts. We shall not often be astray if we draw a conventional line at .05 …”
Lecture 5: Statistical hypothesis testing
General procedure for H testing:
- Specify Null (Ho) and alternate (Ha)
- Determine test (and test statistic) to be used
- Test statistic is used to compare your data to expectation under Ho (null hypothesis)
- Specify significance (α or p value) level below which Ho will be rejected
asdfasfasd
Lecture 5: Statistical hypothesis testing
General procedure for H testing: - Collect data - Perform test - If p-value < α, conclude Ho is likely false and reject it - If p-value > α, conclude no evidence Ho is false and retain it
asdfasfasd
Lecture 5: Brief review
Recall… - Major goal of statistics: inferences about populations from samples… and assign degree of confidence to inferences - Statistical H-testing: formalized approach to inference - Relies on specifying null hypothesis (Ho) and alternate hypothesis (Ha) - Tests assess likelihood of the null hypothesis being true - Expressed as p-value: probability of obtaining sample value of statistic (or more extreme one) if Ho is true
asdfasfasd
Lecture 5: Brief review
Recall hospital example - Probability of getting sample like A (with ȳ at least as far away from 3700 as 3500)? - p(ȳ ≤ 3500 or ȳ ≥ 3900)
What about - 1-tailed or 2-tailed test?
Can solve using SND and z-scores
asdfasfasd
Lecture 5: Brief review
z= (3500-3700)/410 = -0.48
- From z table: p= 0.3156 X 2
- p of getting sample as far away from µ as A is = 0.6312 (63.1%)
But- usually can’t use z!
Can use t-distribution instead…
Lecture 5: H test for a single population
Let’s rephrase the question somewhat: What is probability that sample A (ȳ=3500, s=676) is from population with µ = 3700 (and σ = ?)?
Lecture 5: H test for a single population
This is a 1-sample test - 1-sample tests ask whether - µ is different than some number (e.g., 0 or 3700 or whatever) - based on sample - Another way: - probability this sample came from population with certain µ (mean) (e.g., 3700)
Lecture 5: H test for a single population
Common approach is one-sample t-test, which uses t statistic:
- St is value of statistic of interest from sample (e.g., ȳ) - θ is population value if Ho is true (e.g., µ=0) - SSt is standard error of the mean (s/√n)
What does equation tell us about relationships between variables?
\(t_s = \frac{St - \theta}{S_{St}}\)
test statistic (ts) is calculated by taking difference between observed statistic and value you’d expect under Ho then dividing by the standard error.
This standardization allows you to measure how many “standard errors away” your result is from what would be expected if the null hypothesis were true.
Lecture 5: H test for a single population
Process:
Specify Ho (e.g., µ = 0) and HA (e.g., µ ≠ 0)
Specify significance level (e.g., α = 0.05)
Take sample from population
Calculate:
\(t = \frac{\bar{y} - 0}{s/\sqrt{n}}\)
Lecture 5: H test for a single population
If sample from pop with mean or µ = 0 then t will be close to 0
|Large| t values are more likely if Ho false
Compare t with t-distribution: - if t is further than t at specified significance level (p = 0.05) (σ) - t has less than 5% chance of coming from null distribution
\(t_s = \frac{St - \theta}{S_{St}}\)
Lecture 5: H test for a single population
Is birth weight in population significantly different than 3700 g? - Specify Ho: µ = 3700 and Ha: µ ≠ 3700 - Specify significance level (α = 0.05) - Take sample from population: ȳ = 3500, s = 676
- “What is probability of getting sample at least as far from µ as 3500?”
asdfasfasd
Lecture 5: H test for a single population
Is this one-tailed or two-tailed question?
Calculate t = (3500 – 3700) / (676/ √ 20)
t = -1.32
Compare with t-distribution for df=19
Lecture 5: H test for a single population
~0.10 of t distribution left of t=-1.32 and ~0.1 right of 1.32 (~0.2 overall)
Probability of getting a sample as extreme (or more) as this is ~0.2
Lecture 5: H test for a single population
- (p=0.203 to be exact)
- No sufficient evidence to reject Ho
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd
Lecture 5: H test for a single population
asdfa
asdfasfasd